{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dmq3S-VSy6MT"
   },
   "source": [
    "# **`cruw-devkit` Usages for ROD2021 Challenge**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a simple tutorial of `cruw-devkit` for the [ROD2021 Challenge](https://www.cruwdataset.org/rod2021). Please find the source code at https://github.com/yizhou-wang/cruw-devkit.\n",
    "\n",
    "The tutorial is written by [Yizhou Wang](http://yizhouwang.net/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-SwZfwvg2Xw5"
   },
   "source": [
    "## CRUW Dataset Object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "26XH9yRJqyzE"
   },
   "source": [
    "The CRUW dataset object includes the sensor configurations, calibration parameters, object configurations, and coordinate mappings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cruw import CRUW\n",
    "data_root='/mnt/disk1/CRUW/ROD2021'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### ROD2021 Dataset File Structure\n",
    "\n",
    "Download [ROD2021 dataset](https://www.cruwdataset.org/download#h.mxc4upuvacso).\n",
    "We put the data files as the following structure using [this script](https://github.com/yizhou-wang/cruw-devkit/tree/master/scripts/reorganize_rod2021.sh).\n",
    "\n",
    "```\n",
    "data_root\n",
    "  - sequences\n",
    "  | - train\n",
    "  | | - <SEQ_NAME>\n",
    "  | | | - IMAGES_0\n",
    "  | | | | - <FRAME_ID>.jpg\n",
    "  | | | | - ***.jpg\n",
    "  | | | - RADAR_RA_H\n",
    "  | | |   - <FRAME_ID>_<CHIRP_ID>.npy\n",
    "  | | |   - ***.npy\n",
    "  | | - ***\n",
    "  | | \n",
    "  | - test\n",
    "  |   - <SEQ_NAME>\n",
    "  |   | - RADAR_RA_H\n",
    "  |   |   - <FRAME_ID>_<CHIRP_ID>.npy\n",
    "  |   |   - ***.npy\n",
    "  |   - ***\n",
    "  | \n",
    "  - annotations\n",
    "  | - train\n",
    "  | | - <SEQ_NAME>.txt\n",
    "  | | - ***.txt\n",
    "  | - test\n",
    "  |   - <SEQ_NAME>.txt\n",
    "  |   - ***.txt\n",
    "  - calib\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "X0nQ1r0iq3YY"
   },
   "outputs": [],
   "source": [
    "dataset = CRUW(data_root=data_root, sensor_config_name='sensor_config_rod2021')\n",
    "print(dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Camera and radar configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ozezyj1NxiCK"
   },
   "outputs": [],
   "source": [
    "print(dataset.sensor_cfg.camera_cfg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(dataset.sensor_cfg.radar_cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calibration parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('camera intrinsics:')\n",
    "print(dataset.sensor_cfg.calib_cfg['cam_calib']['2019_04_09']['cam_0']['camera_matrix'])\n",
    "print('camera to radar translation:')\n",
    "print(dataset.sensor_cfg.calib_cfg['t_cl2rh'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Object classes of interest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(dataset.object_cfg.classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mapping between range/azimuth and the indices in RF images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cruw.mapping import ra2idx, idx2ra\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Map from absolute range (m) and azimuth (rad) to RF indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rng = 5.0\n",
    "azm = math.radians(30)  # degree to radians\n",
    "rid, aid = ra2idx(rng, azm, dataset.range_grid, dataset.angle_grid)\n",
    "print(rid, aid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Map from RF indices to absolute range (m) and azimuth (rad)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rid = 20\n",
    "aid = 95\n",
    "rng, azm = idx2ra(rid, aid, dataset.range_grid, dataset.angle_grid)\n",
    "print(rng, math.degrees(azm))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: The range and azimuth cannot be absolutely recover due to the discretization from absolute range/azimuth values to RF pixels."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nUazqhSfxu86"
   },
   "source": [
    "## Data Visualization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from cruw.visualization.examples import show_dataset_rod2021\n",
    "\n",
    "def get_paths(seq_name, frame_id):\n",
    "    image_path = os.path.join(data_root, 'sequences', 'train', seq_name, \n",
    "                              dataset.sensor_cfg.camera_cfg['image_folder'], \n",
    "                              '%010d.jpg' % frame_id)\n",
    "    chirp_path = os.path.join(data_root, 'sequences', 'train', seq_name, \n",
    "                              dataset.sensor_cfg.radar_cfg['chirp_folder'],\n",
    "                              '%06d_0000.npy' % frame_id)\n",
    "    anno_path = os.path.join(data_root, 'annotations', 'train', seq_name + '.txt')\n",
    "    return image_path, chirp_path, anno_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_name = '2019_04_09_BMS1000'\n",
    "frame_id = 400\n",
    "image_path, chirp_path, anno_path = get_paths(seq_name, frame_id)\n",
    "show_dataset_rod2021(image_path, chirp_path, anno_path, dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_name = '2019_04_09_PMS3000'\n",
    "frame_id = 200\n",
    "image_path, chirp_path, anno_path = get_paths(seq_name, frame_id)\n",
    "show_dataset_rod2021(image_path, chirp_path, anno_path, dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_name = '2019_05_29_PBMS007'\n",
    "frame_id = 300\n",
    "image_path, chirp_path, anno_path = get_paths(seq_name, frame_id)\n",
    "show_dataset_rod2021(image_path, chirp_path, anno_path, dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_name = '2019_09_29_ONRD001'\n",
    "frame_id = 834\n",
    "image_path, chirp_path, anno_path = get_paths(seq_name, frame_id)\n",
    "show_dataset_rod2021(image_path, chirp_path, anno_path, dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_name = '2019_09_29_ONRD002'\n",
    "frame_id = 900\n",
    "image_path, chirp_path, anno_path = get_paths(seq_name, frame_id)\n",
    "show_dataset_rod2021(image_path, chirp_path, anno_path, dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "seq_name = '2019_09_29_ONRD011'\n",
    "frame_id = 1000\n",
    "image_path, chirp_path, anno_path = get_paths(seq_name, frame_id)\n",
    "show_dataset_rod2021(image_path, chirp_path, anno_path, dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation Script\n",
    "\n",
    "An example for the usage of evaluation tool used in the ROD2021 Challenge is shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cruw.eval import evaluate_rod2021\n",
    "\n",
    "submit_dir = '<SUBMISSION_DIR>'\n",
    "truth_dir = '<ANNOTATION_DIR>'\n",
    "evaluate_rod2021(submit_dir, truth_dir, dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "cruw-devkit tutorial.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}